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METHOD FOR INCREASING THE PERFORMANCE OF A 
TRAFFIC SIGN RECOGNITION SYSTEM 
/\-e>C^OVsG^£UKO 6C TWIT >KN5MTi^N 

[0001] The invention concerns a process for improving the 

performance of a system for recognizing and/or indicating traffic 

signs according to the precharacterizing portion of Patent Claims 

1 and 10. 

[0002] Documents DE 3619824 C2 and DE 198 52 631 Al disclose 
processes and devices which recognize, classify and cause to be 
displayed traffic signs extracted from images of traffic scenes. 
Herein the processes analyze the image data provided by image 
sensors without any pre -recognition regarding the actual 
scenario. For this reason these known processes are time 
consuming and require a high computer capacity and performance in 
^ order to be able to process this data in real time. 

[0003] The task of the invention is thus to provide a process and a 
kj device, which enhances the performance of previously known 

systems for recognizing and displaying the contents of traffic 
signs. To accomplish this, on the one hand, the computer 



capacity and performance required for the real time operation 
must be reduced and, on the other hand, the display quality of 
the information contents of recognized traffic signs must be 
improved . 

[0004] The task is solved by a process and a device, in which a 
system for recognizing and/or indicating traffic signals, the 
system including a camera and a therewith associated evaluation 
unit for image recognition and/or for image display, utilizes a 
map-based navigation system and/or information originating from a 
traffic information system in the recognition and/or the display 
of the content of traffic signs. 
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[0005] In one conceivable embodiment of the invention the 
communication between the system for recognizing and/or display 
of traffic signs and the map-based navigation system or as the 
case may be the traffic information system occurs via a vehicle 
on-board data bus . 

[0006] Preferably, the system for detection of traffic signs can be 
switched to enhanced performance when, on the basis of 
information as could be supplied by the map-based navigation 
system and/or the traffic information system, it can be 
determined that the vehicle is passing through an area in which a 
reliable detection of traffic signs may be problematic. Examples 
of such areas in which caution is to be taken include transition 
areas for entry and departure of residential areas. There signs 
can be found indicating entry of a residential area and departure 
from a residential area, which imply speed limitations or, as the 
case may be, lifting of the speed limitations. These signs are 
however not round as conventional, but rather square. The signs 
are easily confused with commercial signs, particularly when 
traffic sign recognition is carried out using monochrome cameras, 
since here the relatively significant yellow coloration of the 
signs cannot be used as a recognition criteria. A further area 
in which particular care is to be taken is the environment of 
traffic influencing- facilities and change or detour signs. Here 
one must presume special display and arrangement of the traffic 
signs, such as for example an intense representation of the black 
and white parts of the traffic sign as provided for improved 
night visibility. There is also an increased possibility of a 
suddenly occurring change or switching on of a traffic sign 
symbol, for example, sudden speed reduction to 120 km/h on 
highways. A particular type of traffic sign is a traffic light. 
In principle these can be classified with signs which change, 
since the condition of the display can change and therewith an 
increased observance is required. Using map-based information it 
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is however possible to operate the system with increased 
recognition performance for recognition of traffic signs in areas 
of traffic lights (intersections, train crossings) . 

[0007] While the above examples were concerned principally only 
with interaction with information from map-based navigation 
systems, examples of situations will be presented below in which 
increased recognition performance is required and which are 
advantageously associated with information originating from 
traffic information systems. This is essentially information 
advising that poor visibility due to fog or rain exists in 
certain regions traveled by the vehicle. In such a situation 
there is a presumption of increasingly poor contrast in the image 
data, so that the system for traffic sign recognition preferably 
must enhance the contrast improvement of the images. Preferably 
yp information regarding traffic jams is also taken into 

S consideration, since here advanced notification can be given to 

M the observer, in the form of the display of symbols of a traffic 

m Z jam, a useful distance in advance of the traffic jam, without 

fg even having to pass such a traffic sign. 

\r\ [0008] By this inventive possibility of temporarily varying the 

N 5 performance of the traffic sign recognition system, this opens 

M the possibility of economically realizing a system for 

H= recognition of traffic signs. It is also advantageous that it is 

not necessary to provide a system, which in normal operating 
conditions is inadequately tasked and only in critical situations 
is completely occupied. Rather, the possibility is presented of 
using modern mult i -processor data processing units taking 
advantage of the internal vehicle data communications and the 
number of available data processors present in modern vehicles to 
utilize the temporarily free, unutilized computer capacity in 
productive economic employment. 
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[0009) A further advantageous design characteristic of the 
invention is comprised therein, that information regarding the 
type and/or the position and/or the condition and/or the 
visibility and/or the size of a recognized traffic sign can be 
stored in an data storage unit. This data storage unit can 
either be associated with a map-based navigation system or, as 
the case may be, a traffic information system, or alternatively 
may be a component of the system for traffic sign recognition 
and/or display. By the integration of this data storage unit the 
observer can be alerted, via an acoustic or optical signal, to 
the fact that along a previously traveled path a change in 
respect to the position or the presence or absence of a vehicle 
sign has occurred. Thereby it is possible to alert the observer 
to, for example, construction- site- specific traffic regulations 

p or, in the case of changed right-of-way, regulations in a 

crossing situation. It is further of advantage when, in the same 

h] manner, a warning signal is provided to the observer when the 

H system for traffic sign recognition does not recognize an actual 

tf J traffic sign or cannot classify it adequately. 

ni 

^ [00010] The invention further provides benefits, when on the basis 

Ul of the map-based position determination the possibility is 

presented to initiate or allow a program for use of a decision 
□ logic which limits the display a traffic sign for such a time, 

until a predetermined road distance has been traveled. This 
predetermined road distance is preferably classification specific 
and can be supplied using a storage unit or hard drive. For 
example in Germany the changing of streets is implicitly 
associated with the lifting of restrictions, such as speed limit 
restrictions. In addition, speed limits must be repeated at 
least every 3 km otherwise they are presumed to be rescinded. By 
this characteristic according to the invention it is achieved 
that a traffic sign can be displayed only as long as absolutely 
necessary even without explicit rescinding. 
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[00011] The invention has particularly advantageous benefits in 
combination with a system for recognition of traffic signs, in 
which the image data of the image sensor is analyzed and 
classified in an information-processing unit and, on the basis of 
the result, is stored in memory as image data representing class - 
specific image segments and/or symbolic figures, which are then 
displayed by means of the display unit. Therein, in the 
framework of the analysis by the information-processing unit, it 
is determined whether the image data of the image sensor contains 
one or more objects, which with sufficiently high probability are 
traffic signs. Consequently the so-detected objects are 

forwarded to a further processing and classification step. In a 
particularly advantageous manner, in the framework of the 
invention, during the further processing of the image data of the 
objects, a separation is undertaken into the traffic sign 
specific upper classes and lower classes. Thereby appropriate 
classification-specific characteristic data are extracted and 
relayed to a separate classification. Depending on whether the 
classification specific characteristic data are correctly 
recognized with high probability during the classification, they 
are replaced by the corresponding or appropriate symbolic image 
representing image data stored in the storage unit or by the 
appropriate original image data stemming from the image sensor. 
The image data resulting from the dissections are recombined into 
a synthetic image of a traffic sign and this image is then stored 
in the memory unit and brought to the display by means of the 
display unit. 

[00012] An important advantage of the invention is comprised 
therein, that it is no longer necessary to carry out the computer 
taxing and with high error quotient associated problem of 
classification of a relatively complex traffic sign which must be 
solved over its entirety, rather the problem, by separation of 



the detected objects into traffic sign specific upper classes and 
lower classes in a preceding classification process, is dissected 
or redirected to clearer and therewith simpler classification 
problems. Therewith in like measure the demands on the necessary 
computing power and the image quality are reduced. 

[00013] A brief example will now be given using a speed limit sign 
for the separation of the traffic sign into traffic sign specific 
upper classes and lower classes. The traffic sign specific upper 
class of this traffic sign belongs to the class of regulating 
signs, which have a characteristic round shape with an internal 
symbol therein. In this special cases of a traffic speed 
limiting sign, the integrated symbol is a number which is 
circumscribed by a broad, red ring. In the case of this traffic 
speed limit indicating sign the internally lying symbol, that is 
the indication as to the speed to which a vehicle speed is to be 
limited, is an element of the traffic sign specific lower class. 

[00014] In the system for traffic sign recognition, with which 
advantageously the invention for improving the performance of a 
system for recognition and/or display of traffic signs can be 
combined, it should now be possible with comparatively low 
computer power and with low equipment expense (for example, a 
monochrome camera instead of a high resolution color camera) to 
check in real time whether the object of a particular shape 
within the detected or observed image is an actual traffic sign. 
If this is the case, then the classification specific 
characteristic data of the object are supplied to a separate 
classification for identification of the traffic sign. Generally 
a hierarchically organized classifier is employed for 
identification of the traffic sign. The classification occurs 
therein in multiple steps. For this, first the characteristic 
data or features associated with the upper class of the object 
are supplied to the classification. If a classification occurs 
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successfully (that is, the class could be correctly recognized 
with high confidence) , then the characteristic data are replaced 
by the appropriate symbolic representation of the object stored 
in the memory unit. If the classification is unsuccessful (that 
is, the class could not be recognized with high confidence) , then 
the characteristic data of the upper class and the characterizing 
data of the lower class are replaced by the original image data 
originally stemming from the image sensor and the image 
classification process is ended. A further step within the 
classification procedure is initiated when the classification of 
the upper class was successful. In this case, the characterizing 
data belonging to the lower class of the object are subsequently 
supplied to a classifier. In the case of successful 

classification of this lower class (that is, the class could be 
recognized with high confidence) , then the characterizing data 
are replaced by the appropriate or corresponding symbolic 
representation of the lower class stored in the memory unit. On 
the other hand, in the case of unsuccessful classification (that 
is, the class could not be recognized with high confidence) the 
characterizing data of the lower class is replaced by the 
corresponding original image data stemming from the image sensor. 

[00015] With this hierarchical process a low error rate traffic 
sign recognition is provided. Therein both hierarchical steps 
for identification of the upper class or as the case may be the 
identification of the lower class may themselves again be 
hierarchically organized. With respect to the design of the 
classifiers employed within the hierarchical classification it is 
conceivable to so design these, that they make decisions on the 
basis of a comparison of the distance of the characterizing data 
supplied to them from the class typical characterizing data sets 
stored in the memory unit. Thereby the traffic sign identifying 
object is assigned to that respective upper class, to which the 
distance in the measurement space is the lowest. Thereby it is 



possible to so define a tolerance threshold which is compared 
with a minimal distance value, wherein the distance value falling 
below the tolerance threshold serves as indicator to a high 
probability of correct occurrence of recognition. In another 
advantageous design it is also possible to so design the 
classifiers, that they can be trained with the aid of a traffic 
sign specific learning succession of passes. This process is 
particularly advantageous, since the quality of the image data 
delivered by the image sensor depends strongly upon the 
environmental factors such as weather and light relationships. 



